计算机与现代化 ›› 2012, Vol. 198 ›› Issue (2): 26-30.doi: 10.3969/j.issn.1006-2475.2012.02.008

• 算法设计与分析 • 上一篇    下一篇

基于MapReduce的ID3决策树分类算法研究

钱网伟   

  1. 同济大学电子与信息工程学院,上海201804
  • 收稿日期:2011-10-21 修回日期:1900-01-01 出版日期:2012-02-24 发布日期:2012-02-24

Research on ID3Decision Tree Classification Algorithm Based on MapReduce

QIAN Wang-wei   

  1. School of Electronics and Information Engineering, Tongji University, Shanghai 201804, China
  • Received:2011-10-21 Revised:1900-01-01 Online:2012-02-24 Published:2012-02-24

摘要: 决策树算法是经典的分类挖掘算法之一,具有广泛的实际应用价值。经典的ID3决策树算法是内存驻留算法,只能处理小数据集,在面对海量数据集时显得无能为力。为此,对经典ID3决策树生成算法的可并行性进行了深入分析和研究,利用云计算的MapReduce编程技术,提出并实现面向海量数据的ID3决策树并行分类算法。实验结果表明该算法是有效可行的。

关键词: 云计算, 数据挖掘, 决策树, ID3, MapReduce

Abstract: Decision tree is widely used in data mining which is one of the typical classification algorithms. Traditional ID3 tree learning algorithms require training data to reside in memory on a single machine, so they cannot deal with massive datasets. To solve this problem, this paper analyzes the parallel algorithm of ID3 decision tree based on MapReduce model, then proposes a parallel and distributed algorithm for ID3 decision tree learning. The experimental results demonstrate the algorithm can scale well and efficiently process large-scale datasets on commodity computers.

Key words: cloud computing, data mining, decision tree, ID3, MapRedue

中图分类号: